FLUSH: A Flexible Lexicon Design

Approaches to natural language processing that use a phrasal lexicon have the advantage of easily handling linguistic constructions that might otherwise be extragrammatical. However, current phrasal lexicons are often too rigid: their phrasal entries fail to cover the more flexible constructions. FLUSH, for Flexible Lexicon Utilizing Specialized and Hierarchical knowledge, is a knowledge-based lexicon design that allows broad phrasal coverage. I. I n t r o d u c t i o n Natural language processing systems must use a broad range of lexical knowledge to account for the syntactic use and meaning of words and constructs. The problem of understanding is compounded by the fact that language is full of nonproductive constructs--expressions whose meaning is not fully determined by examining their parts. To handle these constructs, some systems use a phrasal lexicon [Becket, 1975, Wilensky and Arena, 1980b, Jacobs, .1985b, Steinacker and Buchberger, 1983, Dyer and Zernik, 1986], a dictionary designed to make the representation of these specialized constructs easier. The problem that phrasal lexicons have is that they are too rigid: the phrasal knowledge is entered in a way that makes it difficult to represent the many forms some expressions may take without treating each form as a distinct "phrase". For example, expressions such as "send a message", "give a hug", "working directory", and "pick up" may be handled as specialized phrases, but this overlooks similar expressions such as "give a message", "get a kiss", "working area", and "take up". Specialized constructs must be recognized, but much of their meaning as well as their flexible linguistic behavior may come from a more general level. A solution to this problem of rigidity is to have a hierarchy of linguistic constructions, with the most specialized phrases grouped in categories with other phrases that behave similarly. The idea of a linguistic hierarchy is not novel, having roots in both linguistics [Lockwood, 1972, Halliday, 1978] and Artificial Intelligence [Sondheimer et al., 1984]. Incorporating phrasal knowledge into such a hierarchy was suggested in some AI work [Wilensky and Arena, 1980a], but the actual implementation of a hier186 archical phrasal lexicon requires substantial extensions to the phrasal representation of such work. The Flexible Lexicon Utilizing Specific and Hierarchical knowledge (FLUSH) is one component in a suite of natural language processing tools being developed at the GE Research and Development Center to facilitate rapid assimilation of natural language processing technology to a wide variety of domains. FLUSH has characteristics of both traditional and phrasal lexicons, and the phrasal portion is partitioned into four classes of phrasal entries: